Spin-orbital entanglement and frustration in pyrochlores ^2Mo207 
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Electronic and magnetic properties of molybdenum pyrochlores A2M02O7 are studied by the 
fully relativistic density- functional theory plus on-site repulsion {U) method, with focusing on the 
insulating material Y2M02O7. As U increases, the ground state turns from a ferromagnetic metal 
into an antiferromagnetic insulator. In the insulating phase, the system exhibits peculiar magnetic 
degeneracy, which suggests strong anisotropy in the effective magnetic interactions. Analyzing a 
three-orbital Hubbard model, we show that the degeneracy is not solely in spin but in the spin-orbital 
entangled object formed under the cooperation of the trigonal distortion. Coulomb interactions, and 
spin-orbit interaction. The orbital frustration in the Ad^ electronic configuration plays an important 
role. The results give a new insight into the spin-glass behavior in the insulating materials. 
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Spin and orbital degrees of freedom of electrons play a 
crucial role in strongly correlated electron systems. The 
two degrees of freedom are entangled with each other 
via strong Coulomb interactions and the relativistic spin- 
orbit interaction (SOI) [1]. In general, the entanglement 
through the former mechanism is important in tran- 
sition metal compounds, while the latter is dominant in 
bd systems. The spin-orbital entanglement is a source 
of fascinating and intricate properties, such as compli- 
cated spin-orbital orderings [2] and topologically nontriv- 
ial states [3]. Meanwhile, further intriguing situation is 
brought about by geometrical frustration of the lattice 
structure [4, 5]. Frustration suppresses a simple-minded 
ordering, and the residual spin and orbital fluctuations 
can be an origin of interesting phenomena, such as heavy- 
fermion behavior and exotic orders. 

A family of pyrochlore oxides A2B2O7 is a model sys- 
tem for studying the effects of spin-orbital entanglement 
and geometrical frustration [6]. In particular, compounds 
with 5=Mn, Mo, Ir, and Os are interesting as they 
exhibit a metal-insulator transition (MIT) by changing 
temperature (T), pressure, and A-site cations. For in- 
stance, in 3d systems with 5=Mn, the importance of 
Coulomb interactions has been argued for the mecha- 
nism of MIT and giant magnetoresistance [7, 8] . On the 
other hand, for 5d pyrochlores with B=1t and Os, recent 
first-principles studies revealed that SOI plays a dom- 
inant role in determining their peculiar electronic and 
magnetic properties [9, 10]. 

Mo pyrochlores A2M02O7 are of particularly interest 
as Mo 4:d electrons are subject to both strong Coulomb 
interactions and SOI. The system exhibits MIT by A-site 
substitution [11-13] as well as external pressure [14, 15]. 
The compounds with relatively large A-site ionic radii, 
e.g., Nd and Sm, show ferromagnetic (FM) metallic be- 
havior at low T, while those with smaller ionic radii, such 
as Y, Dy, and Tb, are insulating and exhibit a spin-glass 



(SG) transition instead of conventional long-range order- 
ing [16-20]. A Hartree-Fock approach using a realistic 
band structure showed that Coulomb repulsions play a 
central role in the metal-insulator transition and the com- 
petition between magnetic interactions [21]. The role of 
orbital ordering in the competition was studied by ex- 
tended double-exchange models [22-24]. The SG behav- 
ior was also studied by a Heisenberg spin model coupled 
with local lattice distortions [25, 26]. In the previous 
studies, however, unified understanding of MIT and pe- 
culiar SG insulating state has not been reached yet. In 
particular, SOI was not treated on an equal footing to 
Coulomb interactions, though it will be crucial in the 
spin-orbital entanglement in the Ad system. 

In this Letter, we investigate the electronic and mag- 
netic properties in 742M02O7 by the fully relativistic lo- 
cal spin density approximation(LSDA)+/7 method. We 
find that, by increasing the electron correlation, the sys- 
tem exhibits MIT from a FM metal to an antiferromag- 
netic (AFM) insulator. Remarkably, the insulating phase 
retains peculiar magnetic degeneracy, which is not ex- 
plained by a simple Heisenberg antiferromagnetic model. 
Analyzing the origin of the degeneracy by a generalized 
spin model and a multi-orbital Hubbard model, we re- 
veal the crucial role of spin-orbital entanglement due to 
cooperative effects between SOI, Coulomb interactions, 
and trigonal distortion of MoOe octahedra. 

We perform density-functional calculations with our 
computational code, QMAS (Quantum MAterials Sim- 
ulator) [27], using the projector augmented- wave 
method [28] and the LSDA+/7 method [29-34] with a 
planewave cutoff energy of 40 Ry. The relativistic effect 
including SOI is fully considered using two-component 
wave functions [33, 34]. The following calculations were 
done for a typical insulating material Y2M02O7 us- 
ing the experimental lattice structure: the lattice con- 
stant a = 10.21 A [35] and the so-called u parameter 
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FIG. 1. (color online), (a) Cubic unit cell of a pyrochlore lattice, composed of Mo atoms in A2Mo207. The arrows represent the 
Mo spin moments in the 4in/4out magnetic structure, which are along the local [111] axes cx}^^ . The circle denotes a primitive 
four-site unit cell, whose enlarged figure shows the local coordinates xyz and the vectors cXij in the effective spin model in 
Eq. (1). The energy diagram shows the trigonal field splitting of t2g orbitals. See the text for the details, (b) LSDA+[/ results 
for [/eff dependences of energies for various q = magnetic states. The magnetic patterns are shown in (c). Energy is measured 
from that of the 4in/4out state. The arrows for the data at Uef£ — 4 eV denote the energies calculated by the model (1). In 
the 4in/4out state in (c), small gray arrows denote the DM vectors Ckfj^ in the model (1). 



x(Oi) = 0.33821 [36]. Every MoOe tetrahedron is com- 
pressed along the local [111] axis (trigonal distortion) for 
^(Oi) > 0.3125 [see Fig. 1(a)]. We adopt a primitive unit 
cell with four Mo atoms. Experimentally, the magnetic 
and electronic properties vary systematically with the A- 
site ionic radius [11-13], which is regarded as the band- 
width control, namely, the control of electron correlation. 
We discuss such a systematic change by controlling the 
strength of electron correlation, t/gfr {= U — J) [37], in 
the following. 

Figure 1(b) shows the energies of various types of gr = 
magnetic structures as functions of the Coulomb repul- 
sion Uef[. Here we performed self-consistent calculations 
with constraints on the directions of Mo spin moments 
depending on each magnetic structure shown in Fig. 1(c). 
The system exhibits MIT at /7eff 3 eV from a FM metal 
to an AFM insulator while increasing [/efr; the density of 
states (DOS) is shown in Fig. 2(a). Although MIT is 
consistent with the trend in the A-site substitution in 
A2M02O7, the insulating state is experimentally SO in- 
stead of showing the AFM order. This discrepancy is 
discussed in detail below. 

In the insulating region for /7eff ^ 3 eV, the 4in/4out 
order is the most stable among the magnetic structures 



considered [see Figs. 1(a) and 1(c)]. The Mo spin mo- 
ment is 1.54 fiB at /7eff = 4 eV, indicating that a 
S = 1 spin is formed in the insulating phase by the 
Hund's coupling [Fig. 1(a)]. However, there are many 
other low-energy states, with their relative energies be- 
ing almost independent of [/eff. In particular, the 3in- 
lout and 2in-2out states are almost degenerate with 
the 4in/4out ground state. This observation implies a 
substantial local [111] easy-axis anisotropy, similarly to 
the case of Cd20s207 [10]. However, a simple S = 1 
AFM spin model with the easy-axis anisotropy cannot 
account for the energy spectrum in Fig. 1(b). For ex- 
ample, FM [111] ([Oil]) is always substantially lower in 
energy than 2up-2down[lll]([011]), contrary to the ex- 
pectation for the AFM model. This strongly suggests the 
existence of further intricate interactions. 



In order to understand the results in Fig. 1(b), we con- 
sider a generalized Heisenberg model including all the 
symmetry-allowed pairwise interactions between nearest- 
neighbor (NN) spins [38, 39]. For simplicity, we restrict 
the consideration to classical spins. The Hamiltonian is 
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written in the form 

^ = E E Jk{Si-oc%){Syoc%) 

{ij) k—a,h,c 

+Jdm E cx?/'-iSixS,)-D^{S.-nl''fil) 

where Jk are anisotropic NN exchange couphngs between 
Mo spins Si and Sj; cx^j are cubic axes on NN bonds 
[see Fig. 1(a)]. We take \Si\ = 1. The second term 
denotes the Dzyaloshinsky-Moriya (DM) interaction be- 
tween NN Mo atoms [40]. The DM vectors ag^ are 
shown in Fig. 1(c). The third term represents the single- 
ion anisotropy along the local [111] axes cxj^^. 

We determine the parameters in the model (1) so as 
to reproduce the energy levels in the insulating region 
in Fig. 1(b). We find that all the levels are well ex- 
plained by taking Ja > 0, Jb < 0, Jc < 0, Jdm < 0, and 

> 0. A typical result of the fitting is shown by the 
arrows in Fig. 1(b); the arrows indicate the energy lev- 
els calculated by the model (1) for {Ja^Jh^Jc^ Jbm^ D) = 
(4.19, -2.94, -4.93, -4.21, 17.80) meV. The values are es- 
timated by fitting the five independent low- lying states, 
the 3in-lout, 2up-2down[001], 2up-2down[011], 3up- 
ldown[lll], and FM[011], at U^e = 4 eV. The results 
indicate that the effective spin model has surprisingly 
anisotropic interactions. 

The degeneracy in Fig. 1(b) is understood by the com- 
petition between the AFM Ja and FM Jb, Jc, and neg- 
ative Jdm- The former Ja favors 4in/4out rather than 
2in-2out and 3in-lout, while the latter three do the op- 
posite. The keen competition indicates that the system is 
in the competing regime between AFM and FM. Indeed, 
we confirmed that the ground state sensitively changes 
from 4in/4out to 2in-2out by a few % modification of 
the model parameters. 
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FIG. 2. (color online), (a) DOS calculated for the ground 
state at UeE — 2.5, 3, and 4 eV. In the data at [/eff = 4 
eV, the contributions from the aig and orbitals are also 
shown, (b) Spin S (red bold arrows) and orbital moments L 
(blue thin arrows) calculated at [/eff = 4 eV. The lengths of 
the arrows are proportional to the moment sizes, while those 
for L are doubled for clarity. 



The analysis clearly shows that not only spin but or- 
bital degree of freedom plays a substantial role. To clarify 
the microscopic origin, we consider the orbital occupation 
by projecting DOS onto aig and e'^ orbital bases [41]: 
laig) = (1, 1, l)/v^ and |e^±) = (e±2-/3, i, e^^-'/^)/Vi, 
respectively, in the basis of dxy^ dyz^ and dzx orbitals. 
The trigonal distortion splits the t2g levels into aig and 
eg^, and each level has a quantized angular momentum 
along the local [111] axis [see Fig. 1(a)]. As shown in 
Fig. 2(a), /7eff splits further the e'^j^ levels, and conse- 
quently, the aig and e'^_ orbitals are almost singly occu- 
pied, while the e'^j^ orbital is mostly unoccupied in the 
insulating phase. Figure 2(b) shows the orbital moments 
L as well as spin moments S calculated for typical low- 
energy states at [/eff = 4 eV. It is obviously seen that L 
is polarized almost along the local [111] direction, being 
coupled with S antiferromagnetically on each Mo atom. 
This strongly suggests that the orbital polarization plays 
a primary role in the anisotropic features in the effective 
spin model through SOI. 
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FIG. 3. (color online), (a) Density distribution of the MLWF 
localized at a Mo site obtained for [/eff=0. We take the [001] 
axis as the quantization axis. The surface coloring represents 
the imaginary part {e^±) and real part (aig) of the wavefunc- 
tions of the majority spin component, respectively, (b) Non- 
magnetic band structure (solid curves). The broken curves 
represent the fit by MLWF. The energy is measured from the 
Fermi level, (c), (d) The NN correlation functions and the 
excitation spectrum of the three-orbital Hubbard model. The 
solid, broken, and dotted horizontal lines in (c) denote the NN 
correlations of L^^^ for the 4in/4out, the 3in-lout, and the 
2in-2out state in the classical limit, respectively. The inset 
shows the orbital off-diagonal (toffdiag) and diagonal (tdiag) 
transfers between eg orbitals. The crosses connected by the 
broken line in (d) denote the 4in/4out energy. 

Now, let us consider the intersite arrangement of the 
e' orbitals. The relative magnitude of the orbital di- 
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agonal and oflF-diagonal transfers (tdiag and toffdiag, re- 
spectively) plays an important role in the orbital or- 
dering [24]; under strong Coulomb interactions, tdiag 
(^offdiag) favors an antiferro (ferro) orbital ordering to 
gain the second-order perturbation energy [see the in- 
set of Fig. 3(c)]. Through the spin-orbital entanglement, 
the AFM 4in/4out state will be favored by ferro orbital 
ordering for |toffdiag| > l^diagl, while FM bonds in Sin- 
lout and 2in-2out by an antiferro orbital arrangement 

for |toffdiag| < l^gl- 

To explicitly demonstrate how the system is affected 
by the competition in the orbit als, we analyze a three- 
orbital Hubbard model for the t2g orbitals by changing 
|^offdiag/Wg|. We use the values of |toffdiag| = 0.145 and 
l^diagl = 0.102 eV obtained by the maximally localized 
Wannier function (MLWF) analysis [42, 43] at Uef^ = as 
a reference [see Figs. 3(a) and 3(b)], and control the ratio 
by taking toffdiag ^ (1 + Q^)^offdiag and tdiag ^ (l-Q^)Wg 
with a control parameter a. We note that |toffdiag/^diag| 
is - 1.4 for 0.33 < x{Oi) < 0.34 [44], indicating that the 
following argument is robust for the series of A2M02O7. 
The Hamiltonian includes the NN hopping term, SOI, 
trigonal splitting, and rotationally symmetric Coulomb 
interactions. We use the trigonal field Atn = 0.25 eV 
and SOI ( = 0.085 eV [45], which are obtained from 
the MLWF analysis. Setting the intraorbital Coulomb 
repulsion U = 4 eV and the Hund's coupling Jh = 0.5 eV, 
we performed perturbative calculations from the limit of 
strong Coulomb interactions up to the lowest order of 
SOI and transfer integrals. 

Figures 3(c) and 3(d) show the NN correlation func- 
tions of L^^^ and S^^^ in the ground state and the cal- 
culated energy spectrum, respectively. Here, L^^^ and 
S^^^ are L and S projected on the local [111] axes, 
respectively. The results clearly show that the ground 
state turns from 2in-2out to 4in/4out by increasing a at 
a = ac — —0.17. For a > ac, the 4in/4out ground state 
is singled out and largely separated from other excited 
states, as shown in Fig. 3(d). While decreasing a, how- 
ever, the gap closes and there appears a large number 
of low-energy states near a ^ ac. Although the MLWF 
estimate a = is in the 4in/4out region, it is close to the 
phase boundary. The result implies that this criticality 
gives the microscopic mechanism for the degeneracy and 
frustration in the spin-orbital entangled objects in Mo 
pyrochlores. 

Our results provide a new insight into the emergence 
of the SG state. The frustration is not solely in spins 
but in the spin-orbital entangled degree of freedom. The 
frustrated spin-orbital entangled objects may be formed 
in the high-T PM phase, and freeze into a glass at low 
T due to inevitable randomness in real materials. Our 
results suggest that the competition between AFM 4in- 
4out and FM 2in-2out originating from orbital frustra- 
tion plays a key role. Recently, the importance of local 
lattice distortions in the SG behaviors was pointed out 



experimentally [46-50] and theoretically [25, 26]. The 
magnetoelastic coupling sensitively affects the degener- 
acy between the different spin-orbital configurations, as 
2in-2out as well as 3in-lout can couple to local vibration 
modes of Mo tetrahedra [51, 52]. We therefore anticipate 
that these two configurations are energetically favored 
relative to AFM 4in/4out, resulting in further frustra- 
tion with enhanced FM correlations. 

To summarize, we found the peculiar magnetic degen- 
eracy in the insulating phase of Mo pyrochlores. Through 
the analyses using the generalized effective spin model 
and three-orbital Hubbard model, we revealed the critical 
role of the spin-orbital entanglement and the degeneracy 
in the orbital sector in the peculiar magnetic properties. 
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